{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "65f3c082-63ef-4f91-bfa5-3d2bf2d84c62",
   "metadata": {},
   "source": [
    "# Understand Standard Modules in a Detector"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71375de8-ba5c-49bb-bc9e-333eaa6553cc",
   "metadata": {},
   "source": [
    "A detector usually consists of:\n",
    "\n",
    "- a backbone, to generate multi-stage feature maps\n",
    "- a feature pyramid network, to fuse features of different stages\n",
    "- a dense prediction head, to predict object at different locations\n",
    "- (optinally) sparse head for two-stage and multi-stage detectors\n",
    "\n",
    "This image is from YOLO v4\n",
    "\n",
    "![](https://miro.medium.com/max/720/1*Z5GOPYFgh7_NTr7drt45mw.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b420800-f32c-4195-9a06-d3cc9d16f55f",
   "metadata": {},
   "source": [
    "In MMDetection, components are defined in `model` key in the config file.\n",
    "\n",
    "In this notebook, we investigate these modules interatively, and understand the connection between Python modules and config files. \n",
    "\n",
    "We use RetinaNet and FCOS as example.\n",
    "\n",
    "Architecture: \n",
    "\n",
    "<img src=\"https://production-media.paperswithcode.com/methods/Screen_Shot_2020-06-23_at_3.34.09_PM_SAg1OBo.png\" alt=\"Architecture\" style=\"width: 800px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bae3fd61-8371-4283-af8c-61644047ef4b",
   "metadata": {},
   "source": [
    "## Backbone\n",
    "\n",
    "We define a ResNet backbone that output feature map C2 to C5.\n",
    "\n",
    "Here index 0 indicates C2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "8fe59c3d-b8ea-4f54-a307-330c99d3b9ad",
   "metadata": {},
   "outputs": [],
   "source": [
    "from mmdet.models.backbones import ResNet\n",
    "\n",
    "backbone = ResNet(depth=18, out_indices=(0,1,2,3))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ab16a77-b8d8-459c-9240-119b565310a8",
   "metadata": {},
   "source": [
    "We forward a random tensor and check the output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "0b2c49cf-7925-478c-8207-e464f4fbe193",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "\n",
    "img = torch.rand(1,3,1000,600)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "eda9ab5d-204a-44bc-9431-b7dd21634f37",
   "metadata": {},
   "outputs": [],
   "source": [
    "bbout = backbone(img)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "4c112978-2159-41e0-b69f-bd45035484da",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([1, 64, 250, 150])\n",
      "torch.Size([1, 128, 125, 75])\n",
      "torch.Size([1, 256, 63, 38])\n",
      "torch.Size([1, 512, 32, 19])\n"
     ]
    }
   ],
   "source": [
    "for o in bbout:\n",
    "    print(o.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "952b11b3-49b5-4129-986c-a8f6f915cb38",
   "metadata": {},
   "source": [
    "These tensors are downsampled from 4x to 32x respectively. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55338895-cf92-4454-bb3a-7ffc57bff36d",
   "metadata": {},
   "source": [
    "## Neck\n",
    "\n",
    "We define a plain feature pyramid network to fuse different modules. \n",
    "\n",
    "`start_level=1` means index 1 from the backbone, which is C3. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "48abfc63-9d84-4ca6-8d8f-668962ae9763",
   "metadata": {},
   "outputs": [],
   "source": [
    "from mmdet.models.necks import FPN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "8e2c5b18-c558-481f-b147-fbf88c4cfc61",
   "metadata": {},
   "outputs": [],
   "source": [
    "neck = FPN(in_channels=[64, 128, 256, 512], start_level=1, out_channels=256, num_outs=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "f0c96a1c-c9c1-4a1a-a5b4-ed38e546e720",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([1, 256, 125, 75])\n",
      "torch.Size([1, 256, 63, 38])\n",
      "torch.Size([1, 256, 32, 19])\n",
      "torch.Size([1, 256, 16, 10])\n",
      "torch.Size([1, 256, 8, 5])\n"
     ]
    }
   ],
   "source": [
    "ncout = neck(bbout)\n",
    "\n",
    "for o in ncout:\n",
    "    print(o.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1cda2a5b-072b-4d25-9212-42e8bce52697",
   "metadata": {},
   "source": [
    "The neck outputs fused features of all 5 levels, with same number of channels. \n",
    "Down-sampling rates ranges from 8x to 128x."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ebab7af5-40ef-4371-8480-0582aa674878",
   "metadata": {},
   "source": [
    "## Anchor-free Heads\n",
    "\n",
    "An anchor-free head slides on the feature map and compute predictions directly (compared to anchor-based heads which compute prediction referenced to anchors)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "9649790c-5227-4225-bf0c-15ba85895da4",
   "metadata": {},
   "outputs": [],
   "source": [
    "from mmdet.models.dense_heads import FCOSHead"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "e7c40443-5db1-430f-b7eb-0845ed727118",
   "metadata": {},
   "outputs": [],
   "source": [
    "head = FCOSHead(num_classes=20, in_channels=256)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "827cdcab-f690-440a-bb75-4c29824ee9d3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "type: <class 'tuple'>\n",
      "length: 3\n"
     ]
    }
   ],
   "source": [
    "hout = head(ncout)\n",
    "\n",
    "print(F\"type: {type(hout)}\")\n",
    "print(F\"length: {len(hout)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "0bc6a519-4124-4601-9022-dce1e90e50f0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([1, 20, 125, 75])\n",
      "torch.Size([1, 20, 63, 38])\n",
      "torch.Size([1, 20, 32, 19])\n",
      "torch.Size([1, 20, 16, 10])\n",
      "torch.Size([1, 20, 8, 5])\n",
      "torch.Size([1, 4, 125, 75])\n",
      "torch.Size([1, 4, 63, 38])\n",
      "torch.Size([1, 4, 32, 19])\n",
      "torch.Size([1, 4, 16, 10])\n",
      "torch.Size([1, 4, 8, 5])\n",
      "torch.Size([1, 1, 125, 75])\n",
      "torch.Size([1, 1, 63, 38])\n",
      "torch.Size([1, 1, 32, 19])\n",
      "torch.Size([1, 1, 16, 10])\n",
      "torch.Size([1, 1, 8, 5])\n"
     ]
    }
   ],
   "source": [
    "for res in hout:\n",
    "    for o in res:\n",
    "        print(o.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6bc9f367-8836-405c-94a5-b5f0d8aca11d",
   "metadata": {},
   "source": [
    "FCOS head predict:\n",
    "\n",
    "- Class probabilities of 20 dimension\n",
    "- Bounding box coodinates of 4 dimension \n",
    "- Centereness of 1 dimension \n",
    "\n",
    "at all positions on all levels of feature maps.\n",
    "\n",
    "\n",
    "During inference, post-processing like thresholding and NMS will produce final detection boxes. \n",
    "\n",
    "During training, predictions will be compared with ground-truth to produce loss"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d83b2855-b4a4-47a1-9e82-4db27d29c139",
   "metadata": {},
   "source": [
    "## Anchor-based Heads\n",
    "\n",
    "Anchor-based head is programmingly more complex as it contains an anchor generation module. Let's investigate this first. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "5ff57bb0-d8a5-4098-aeb4-430bb0f261ea",
   "metadata": {},
   "outputs": [],
   "source": [
    "from mmdet.core.anchor import AnchorGenerator\n",
    "# from mmdet.models.task_modules import AnchorGenerator # for v2.0\n",
    "\n",
    "ag = AnchorGenerator(\n",
    "            scales=[8],\n",
    "            ratios=[0.5, 1.0, 2.0],\n",
    "            strides=[4, 8, 16, 32, 64])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "0ebd8a6c-5eee-40ea-9d33-92d4e3f120cb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[tensor([[-22.6274, -11.3137,  22.6274,  11.3137],\n",
      "        [-16.0000, -16.0000,  16.0000,  16.0000],\n",
      "        [-11.3137, -22.6274,  11.3137,  22.6274]]), tensor([[-45.2548, -22.6274,  45.2548,  22.6274],\n",
      "        [-32.0000, -32.0000,  32.0000,  32.0000],\n",
      "        [-22.6274, -45.2548,  22.6274,  45.2548]]), tensor([[-90.5097, -45.2548,  90.5097,  45.2548],\n",
      "        [-64.0000, -64.0000,  64.0000,  64.0000],\n",
      "        [-45.2548, -90.5097,  45.2548,  90.5097]]), tensor([[-181.0193,  -90.5097,  181.0193,   90.5097],\n",
      "        [-128.0000, -128.0000,  128.0000,  128.0000],\n",
      "        [ -90.5097, -181.0193,   90.5097,  181.0193]]), tensor([[-362.0387, -181.0193,  362.0387,  181.0193],\n",
      "        [-256.0000, -256.0000,  256.0000,  256.0000],\n",
      "        [-181.0193, -362.0387,  181.0193,  362.0387]])]\n"
     ]
    }
   ],
   "source": [
    "bbx = ag.single_level_grid_priors(featmap_size=(125,75), level_idx=1)\n",
    "bbx = ag.gen_base_anchors()\n",
    "print(bbx)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "5e018ebd-fae2-42f0-9ff0-06d7d37c0f42",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\wangruohui\\Miniconda3\\envs\\mm\\lib\\site-packages\\mmengine\\visualization\\visualizer.py:170: UserWarning: `Visualizer` backend is not initialized because save_dir is None.\n",
      "  warnings.warn('`Visualizer` backend is not initialized '\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfQAAAHzCAYAAADW0+8yAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAgP0lEQVR4nO3de2zb9b3/8dfXdmwncZKmTdK0EDp6up0OwqCFnh7KBpwjIfgdncN2pE0bEv+wqRIUDSZA0wG1k4ZUJjS16Exs6qYzHdrtj0mTpoHa9ehwSldYW5VSaFfahNBLEpImqWvn5ksSXz6/Pzh0K4emTuLEzjvPR//CfPz5vn1JnnHs2J5zTgAAYH7zlXoAAAAwcwQdAAADCDoAAAYQdAAADCDoAAAYQNABADCAoAMAYABBBwDAgMAU1vIONAAAzD2vkEU8QgcAwACCDgCAAQQdAAADCDoAAAYQdAAADCDoAAAYQNABADCAoAMAYABBBwDAAIIOAIABBB0AAAMIOgAABhB0AAAMIOgAABhA0AEAMICgAwBgAEEHAMAAgg4AgAEEHQAAAwg6AAAGEHQAAAwg6AAAGEDQAQAwgKADAGAAQQcAwACCDgCAAQQdAAADCDoAAAYQdAAADCDoAAAYQNABADCAoAMAYABBBwDAAIIOAIABBB0AAAMIOgAABhB0AAAMIOgAABhA0AEAMICgAwBgAEEHAMAAgg4AgAEEHQAAAwg6AAAGEHQAAAwg6AAAGEDQAQAwgKADAGAAQQcAwACCDgCAAQQdAAADCDoAAAYQdAAADCDoAAAYQNABADCAoAMAYABBBwDAAIIOAIABBB0AAAMIOgAABhB0AAAMIOgAABhA0AEAMICgAwBgQKDUAwCSNJ4d1/mh88rkMqUeBZiSpuomNVU3yfO8Uo+CBY6goyycGDihf/uff1Mun+MbI+aNbD6rO6+/Uy/e96I8cb9FaRF0lIXkRFI+z6cX7ntBiysXl3ocoCC7TuxS32hfqccAJBF0lBG/59eNi27U0sjSUo8CFKQ+XE/QUTZ4URwAAAYQdAAADCDoAAAYQNABADCAoAMAYABBBwDAAIIOAIABBB0AAAMIOgAABhB0AAAMIOgAABhA0AEAMICgAwBgAEEHAMAAgg4AgAEEHQAAAwg6AAAGEHQAAAwg6AAAGEDQAQAwgKADAGAAQQcAwACCDgCAAQQdAAADCDoAAAYQdAAADCDoAAAYQNABADCAoAMAYABBBwDAAIIOAIABBB0AAAMIOgAABhB0AAAMIOgAABhA0AEAMICgAwBgAEEHAMAAgg4AgAEEHQAAAwg6AAAGEHQAAAwg6AAAGEDQAQAwgKADAGAAQQcAwACCDgCAAYFSD4DSSE4kdWLghMaz46UeRZJ0vP+44um4Dn50UPXh+lKPAxTkTPyM+hJ92n9+v3xeeTw+WrFohW5cdKM8zyv1KJhjnnOu0LUFL0T5O/TRIT37P88q6A+WxTeieDquc0PndEvTLQr5Q6UeByhI51CnEpmEbm68WZ5KH9B0Nq01zWv00gMvlcXXNYqmoDsXj9AXqPHsuIL+oP79//27llQuKfU4OvjRQf3kyE/003/6qRqqGko9DlCQHe/sUEesQ9vu31YWQf/FsV+oc6iz1GOgRAj6AubzfFpSuURLI0tLPYrqw/UK+UNqqGooi3mAQkSCEYUDYTVVN5XFI+KqiqpSj4ASKv09EAAAzBhBBwDAAIIOAIABBB0AAAMIOgAABhB0AAAMIOgAABhA0AEAMICgAwBgAEEHAMAAgg4AgAEEHQAAAwg6AAAGEHQAAAwg6AAAGEDQAQAwgKADAGAAQQcAwACCDgCAAQQdAAADCDoAAAYQdAAADCDoAAAYQNABADCAoAMAYABBBwDAAIIOAIABBB0AAAMIOgAABhB0AAAMIOgAABhA0AEAMCBQ6gHmk/HsuE4MnFByIlnqUWbseP9xxdNxHfzooOrD9aUep+zmmUze5TU6MapcPlfqUcyqDlYr5A+VeoxrOhM/o75En/af3y+fV/rHR+U2z0x4nqdVi1fp+trrSz3KvOE55wpdW/BCq9ovtWvTnk3yeT75PX+px5mReDquc0PndEvTLWXxjbPc5plMOpvWqegpBXyBeX8/KEdj2TE1VTdpRd2KUo9yTZ1DnUpkErq58WZ58ko9TtnNMxPJTFL3/8392nLPllKPUg4KujF5hD4FmVxGuXxOL9z3gm5cdGOpx5mRgx8d1E+O/EQ//aefqqGqodTjlN08k+ke7tb3/ut72nz3ZrU2tZZ6HFOcnDa/sVnX116vJ9c/WepxrmnHOzvUEevQtvu3lUVAy22emdj61lYlJhKlHmNeIehT5HmeFlcu1tLI0lKPMiP14XqF/CE1VDWUxWUpt3kmk8wkFfQHtTSyVC11LaUexxTnnKorqhUJRsr+fiBJkWBE4UBYTdVNZfEr7nKbZybCgXCpR5h35vctDgAAJBF0AABMIOgAABhA0AEAMICgAwBgAEEHAMAAgg4AgAEEHQAAAwg6AAAGEHQAAAwg6AAAGEDQAQAwgKADAGAAQQcAwACCDgCAAQQdAAADCDoAAAYQdAAADCDoAAAYQNABADCAoAMAYECg1AMAc8U5p2w+q5zLzWifidyE8i6vidyExrJjRZoOn8i5nLL57IyvW0+eKvwV8nk8bsHCQNCxYGTzWW07vE3v9b03o32SmaTaLrXpB/t/oPpwfZGmwyeO9R1TVUWVTl08NaN9Kisq9cyGZ9Ta1FqkyYDyRtCxYORcTu/1vafG6kbdvuz2ae8TTUV1Jn5GG1o2aEXdiiJOCCeni8mLWly5WA+semDa+6QyKf329G/VM9JD0LFgEHQsOLcvu12PrHlk2uc/N3hOez/cqwf/9kGtXba2iJPBOaejvUfVUtcyo9soloppd8fuIk4GlD+eXAIAwACCDgCAAQQdAAADCDoAAAYQdAAADCDoAAAYQNABADCAoAMAYABBBwDAAIIOAIABBB0AAAMIOgAABhB0AAAMIOgAABjAx6eiLORdXulsWt3D3UpmkrNyjInchJKZpKKpqM4Nnpv2Pj0jPUpn0+od6dWi8KLiDShpPDuuvMsXdc/ZFAqE5POK+7hgZHxE8XR8RrfR0NiQ0tm0+hP9M9rnWuLpuHL53KztD0wFQUdZGJ0Y1anoKX3vv76noD84K8fIu7zaLrXpTPyM9n64d9r7pLNptV9q1w8P/FA1wZqizZfNZ/Vh/ENN5CbkySvavrPFyWl5zXI1VjUWdd+OWIdCgZCOXTg27T0y+Yzaom166fBL2nl8ZxGnu1L3cLdWLFoh55zmwU0G4wg6ykIun1PAF9DmuzdraWTprBxjIjehH+z/gTa0bNCDf/vgtPfpHenVDw/8UE/d+ZRWN6wu2nyxVExb9m/RQ60P6dbmW4u272zI5XP68aEfa03zGn3j5m8UbV/nnH70px+pOdKsb6/59rT3GR4b1uY3NuuhWx7ShpYNRZvv07Yd2qau4a5Z2x+YCoKOsuH3/GptalVLXcus7D+WHVN9uF4r6lZo7bK1095nUXiRaoI1Wt2wekb7fFp/ol+1oVrd2nyr7v3cvUXbdzZkchm9cvwV3VB3Q1GvA+ecGqsatbxm+Yz2jaViqg3VatXiVUWd79MaqhoIOsoGL4oDAMAAgg4AgAEEHQAAAwg6AAAGEHQAAAwg6AAAGEDQAQAwgKADAGAAQQcAwADeKQ6YA9l8Vj0jPRrLjl11TSwVU3Iiqe7hbrVfap/D6aYul89peHxYA8mBSWf15Kk50qy6cN0cTgcsTAQdmAM9Iz3atGeTEhOJq3462URuQm2X2rT98Paif4pbsTnn1B5rV/uldv2x849XXZfL5/T1m76uJ//+ybkbDligCDowB8ayY0pMJPTMhme0sn7lZ66JpWJ6dt+z2rh2o9Zdt26OJ5yaXD6nzW9s1m3Nt+mbrd+86rrth7crlo7N4WTAwkXQgTni83xaWb9SrU2tn/n/+xP9igQjurH+xquuKReZXEaLwou0vGb5VWd1zhX142UBTI4XxQEAYABBBwDAAIIOAIABBB0AAAMIOgAABvAqd2AasvmsYqmY+hP9Ba2PpWKayE1Mep5oMqqJ3ITi6fg193XOKZPPTHnuQvg8nwK+yb815PI5pbNpjYyPTDprKpNSwBco+HqSpFQ2pbzLF7wewMcIOjBF49lxfRj/UFv2b1FtqLag8yQnkmq71KZn9z2rSDDymWsmchM6efGkXnjrBS2uXDzpfvF0XL2jvZKb8viTcnKqqqjSyvqVV30DHOnjHyjej76vU9FT2nd+31XXtV9qV4WvQicvnix4htPR06oNFna9AvgLgg5MUd7lNZGb0EOtD+nW5lsLOk/3cLe2H96ujWs36sb6Gz9zTTwd1wtvvaCHv/Swbmu+bdL9/vDhH3Sk54g2rdukCn/FVC/CVb3d+7be7HpTT935lKoqqq66LpvP6sWDL6q1sVVfXf3Vz1zjnNPLb7+sSDCiR9Y8UtDxnXP6/uvf13hufFrzAwsZQQemwZOnW5tv1b2fu7eg9e2X2rUovEjrrls36RvLLK5crNuab7vmvmfjZ9U93K2vrf6aQoHQFKef3OnoaX35hi9P+tuHTC6jncd36gtLvnDVWZ1z+n3771UXriv4esq7fMG/9QBwJV4UBwCAAQQdAAAD+JU7UOacc3Jycu4vr4D75L/zLq9cPlfcY31qX8/z5MmT53lFOw6A4iPoQJlzcnrl+Cs62H3w8mkdsQ71jvbqsT2Pye/5i3asruEudcQ69MTeJ1Th+/jFdjWhGj1959NqqWsp2nEAFB9BB8qcc04Huw9qPDeutcvWSpKSmaQSmYRubrxZfl/xgp53efUn+vXFhi8qFAgpMZHQ62dfV+9oL0EHyhxBB+aJtcvW6qk7n5Ik/fLdX0qSnlj/RFFf5f67tt8plo7p8b97XLWhWvWM9OhA54Erft0PoDzxojgAAAzgETpQJpxziqfj+sOHf9DZ+Nm/nC6njliHkpnk5Ufmf+r+k84NntMrx1+54m1aRydGNTo+WtDxAr6AllQtueI5+Hf73lX3cLd+deJXCgfCGhwb1IXEBe3u2K3T0dOX1+VdXmcHz2rNsjUzvdgAioSgA2Uik8+od7RXR3qOqHu4+/Lpzjn1jvYqkUlcPu3c4Dn1J/q17/y+K96i9Xj/cSUnktd869iJ/MfvK3/H8juueCOXnpEeRVNR7e/cr4AvoHQmrXg6rrd739bZwb/6IcM5nR86r3g6XoyLDqAICDpQTpy0ad0mfW311y6flHd5PbbnMd3ceLOeWP+EJOmV469o3/l9+uWDv1TQH7y89un/flq1oVptuXvLpIfpiHXou3u/q63/uFU3Nd50+fRXP3hVu07s0o5/3qGaYI16R3v1nVe/oy33bNH669ZfXpfJZ3TPf97Dc+tAGSHoQJmp8Fdc8UK3XD4nv+eX3+e/fHrAF5DP8ynoDyrgC2ggOaCx7JiSE0n5PJ/6En2THiOaimo8O66LyYtaFF6kxupGRYKRK/YNBUIK+oPyeT5V+K6cycvxd+lAuSHowDw3kBzQpj2bNDQ2pDPxMwr4Ajo5MPmnm6UyKX0Q+0Bb9m9RZaBSD3/pYW28feMcTQxgNhB0YJ4by45paGxIj97xqF5tf1U1oRo9esejk56nc6hTW9/cqmc2PKPXPnhN0VSUX58D8xxBB+ah0YlRHe8/rqf/+2klJ5I6Ez+jV9tfVdulNoX8Ie08vnPS8w+ODWogOaDXPnhNfx74s84NntNAYkBnB89qIDFA3IF5iKAD89Do+KiSE0nVhmrl83wK+AKqCdUo5A8pFAipLlw36fmz+az8Pr8iwYiC/qDCgbDqwnXyeT7F0jHlXX6OLgmAYiHowDy1uHKxtty9RX2JPp0cOKlH73hUO4/vVF24Ts//w/OTnrf9UrtORU/p8XWP62dHf6aWuhY995XntOOdHTp18dQcXQIAxcQ7xQEAYABBBwDAAIIOAIABBB0AAAMIOgAABhB0AAAMIOgAABjA36ED89REfkIdsQ5FU1GlMil1DnVqcGxQ2XxW7ZfaJz3v+cHzl88zNDakcCCs9kvtupi8OEfTAyg2gg7MQwFfQLFUTN/d+12NZ8f1QewDbX1zqwaSA/L7/DoVnfzNYVKZlDpiHXr+wPPqHe1VKBDSe/3vKZqMKuDj2wIwH/GVC8xDS6qW6I7ld2jrP27VxeRFbdm/5fIHrUSCET2+7vFJz9851KnnDzyv577ynH7z/m/UHGnWxts3at+5fdp7Zq/8Pv8cXRIAxULQgXnI7/lVG6rVTY03aVF4kSoDlZc/aCXoD+pnR3826fmHxobUO9qr37z/G0VTUa1ZtkatTa3qiHUo6A/O0aUAUEwEHZjnGqsb9fCXHlY0FdW5wXMKB8JqqWuZ9DzhQFihQEjNkWatWbZG9628b46mBTBbCDowz0WCEW28faOccxpIDKguXKfnvvLcpOdpv9Su9/rf08bbN6q1qXWOJgUwmwg6UEacnN7uffvK05xT13CX8i6v37X9TpL0bt+76hnp0asfvHrFi9jODp6Vz/Npxzs7Jj3OxeRFRZNR7Tu3Tx2xjsunH+k5or5En3Z37FY4EFYsFVM0FdWBrgPqS/RdXpfL5zQ6PlqMiwygSAg6UCZ8nk9VFVV6s+tNnY6evny6c04dsQ71J/oVS8ckSd3D3Yqmotp1Ypd83l/eTmIgMaBYOlbQR6AGfAHtPbP3iufM+xJ9+mj4I/36z79WwBfQWHZMfaN92tOxR4cqD10xUyqTUjgQLsZFB1AEBB0oEwFfQCvrV+qpO5/Sl2/48uXT8y6vJ/Y+oS82fFGP/93Hr17/1YlfaX/nfu345x1XBNk5p7zLF3zMT7+afXfHbv36z7/Wfzz4H4oEI7owekGP7n5UW+7eonXXrbu8LpvPatOeTVpStWS6FxdAkRF0oIx88ii9NlR7+bRcPqcKX4VCgdDl08OBsAK+gGqCNQoFQkU7/if7RoIR1YZqNTI+Ir/n/z8zZXIZVfgq5Mkr2rEBzAxBB+aJxERCPSM9kqTBsUGlM2n1jvYW9c/MYqmYxrJjujB6QSPjI+ob7ZOTK9r+AGYPQQfKnOd5qgnV6PWzr+tA5wFJ0oXEBcXTcX3n1e9c8Rz6TEVTUfWN9unR3Y/K7/nl5OSc47lyYB4g6ECZ8+Tp6TufVu9or5z7+NHy7o7derv3bW25Z4sqfBVFO9aBrgPa07FHW+7eoqqKKkkf/xr+psabinYMALODoANlzvM8tdS1XPFmMaejp3V28KzWX7e+qM+h9yX6dKjykNZdt+6K58wBlD8+PhUAAAMIOgAABvArd2CacvmcMrlMwWudc5Oe55M12Xz2mvvmXV7OOWXyGXm54v3pWKEzZPNZ5V1eOTf5dZB3eeVdvuDryf3vPwBTR9CBaXBy+vGhH+uV468UtH54fFjtsXZtfmOzFoUXfeaadDat96Pv68WDL2rn8Z2T7nd28KzOD53XPf95jzyveEEfHR9VKpPSpj2bJn2xXd7ldbjnsM7Ez+hIz5Grrjt64aiC/qC6hroKOr6T09n4WX1+8eenPDuw0BF0YIpCgZCW1yzXmuY1uqHuhoLOM5AcUPuldt3WfJuW1yz/zDUj4yM6FT2l1sZWfWHJFybdb82yNYqn45df9V5M4UBYS6qWTPqmMTmX05n4Ga2sX6kNLRuuuq5ntEeVgcpJ1/w1J6fu4W41VDVMeW5goSPowBT5PJ8aqxr1jZu/obXL1hZ0nvZL7fpj5x/1zdZvXvXTzfoT/dp3fp++uvqruvdz9xZx4uLL5DI60nNEG1o26LF1j33mGuec2i+1qy5cd9U1n3WekwMnVR2sLua4wILAi+IAADCAoAMAYABBBwDAAIIOAIABBB0AAAN4lTswx671p2bOuVn5c7TZMp9mBSwj6MAc8OQpl89p++HtqgnWfOaaVCal9kvtevntl/X79t/P7YBTlHd5Hb1wVD2jPWq/1H7Vdaejp3XXDXfN4WTAwkXQgTnQHGnW12/6umLp2FXXBHwBVfgqFAlGVBeum8Pppi7v8gr6g6oMVE4661033KX7Vt43h5MBCxdBB+ZAXbhOT/79k5Ou6U/06+TFk3pkzSPz4o1luoa6Jn1jGQBzixfFAQBgAEEHAMAAgg4AgAEEHQAAAwg6AAAGEHQAAAwg6AAAGEDQAQAwgKADAGAA7xSHsuI0+x9MUqxjzNaHqDjnlHf5ou9bTO6v/xXxOvjrPWeyr9PH553tD7r55DhAOSDoKBtj2TFtfmOzqiuqZ2X/nMvpWN8xXUxe1NHeo9PeZ2R8RB2xDv3oTz9SY1Vj0eZLZVM6HT2t77/+fdWGaou272xwcjobP6vu4W6dHDhZ1H3f6npLkWBEPSM9095nLDumU9FTevnoy3rtg9eKNt+nHek9Mmv3V2CqCDrKQnWwWk3VTbq+9npFgpFZOUY2n1VVRZUWVy5WS13LtPeJp+MKBUJqjjRrec3yos2Xd3nVBms1nhsv2p6z6fOLP6+GqgZVB4sXNOfc5Q+nmcltlMqkFA6E1VjVOKN9rqVruEuVgcpZ2x+YCoKOshDyh7SiboWeXP+klkaWzsoxxrJjOnXxlB5Y9YAeWfPItPc5N3hOxy4c07fXfFtrl60t4oRwzqlnpEctdS167ivPTXufWCqmYxeO6Vut39IDqx4o4oRX2nZom05HT8vzvFk7BlAoXhQHAIABBB0AAAMIOgAABhB0AAAMIOgAABhA0AEAMICgAwBgAEEHAMAAgg4AgAEEHQAAAwg6AAAGEHQAAAwg6AAAGEDQAQAwgI9PxYKTyqQUS8Wmff6hsSFl8hkNjw3PaB/8X05OY9mxGd9Gg2ODcnJFnAwofwQdC4YnT5UVlfrt6d9qd8fuae+TzqbVFm3T5jc2qzZUW8QJIUmnoqcUDoR17MKxae/h5JScSCrkDxVxMqC8EXQsGBX+Cj2z4Rn1jPTMaJ/+RL9eOvySHrrlIa1avKpI00GSnHN6+ejLaqxq1LdavzWjvUL+kO5YfkeRJgPKH0HHguHzfGptalVrU+uM9jk3eE47j+/UhpYNWrtsbZGmg/Rx0F/74DW11LXogVUPlHocYF7hRXEAABhA0AEAMICgAwBgAEEHAMAAgg4AgAEEHQAAAwg6AAAGEHQAAAwg6AAAGEDQAQAwgKADAGAAQQcAwACCDgCAAQQdAAADCDoAAAYQdAAADCDoAAAYQNABADCAoAMAYABBBwDAAIIOAIABBB0AAAMCpR5gvsnms9p1Ypfqw/WlHmVGzsTPqHOoUzve2aFIMFLqccpunsnE03F1D3dr26FtaqhqKPU4pjg5Hek9oq7hLm07tK3U41zTga4DGkgMaPvh7fLklXqcsptnJjpiHVrdsLrUY8wrBH0KmqqbdOf1d6pvtE99o32lHmdG+hJ9SmQS6oh1KBwIl3qcsptnMrl8TisWrVDXcJe6hrtKPY451RXVqgxU6nT0dKlHuaaBxICGxobUFm0r9SiSym+emWioatD669aXeox5xXPOFbq24IVWOefkjFwN+8/v14sHX9Suf92lpuqmUo9TdvNcyxS+bjBNnlf+jzC3H96utmibfv4vP5fPK/0zmOU2z0x58ubF/WAOFHQl8Ah9CjzPm/e/xvqEz/N9/MUiryy+8MttnmuycTfADH3y/cDn+crifltu82BucYsDAGAAQQcAwACCDgCAAQQdAAADCDoAAAYQdAAADCDoAAAYQNABADCAoAMAYABBBwDAAIIOAIABBB0AAAMIOgAABhB0AAAMIOgAABhA0AEAMICgAwBgAEEHAMAAgg4AgAEEHQAAAwg6AAAGEHQAAAwg6AAAGEDQAQAwgKADAGAAQQcAwACCDgCAAQQdAAADCDoAAAYQdAAADCDoAAAYQNABADAgUOoBUDrpbFq/OPYLVVVUlXoUnYmfUedQp3a8s0ORYKTU4wAFOdB1QAOJAW0/vF2evFKPo3f73lVlRWWpx0CJEPQFasWiFVrTvEadQ52lHkWS1JfoUyKTUEesQ+FAuNTjAAUZSAxoaGxIbdG2Uo8iSaqsqNRdLXeVxQ8XmHuec67QtQUvRPlzzsmV0U26//x+vXjwRe36111qqm4q9ThAQbYf3q62aJt+/i8/l88rj2cwP4m55xF1Qwq6MXmEvkB5nldWP8X7PJ+8//1XLt8YgWv55GvI5/m436LkuAcCAGAAQQcAwACCDgCAAQQdAAADCDoAAAYQdAAADCDoAAAYQNABADCAoAMAYABBBwDAAIIOAIABBB0AAAMIOgAABhB0AAAMIOgAABhA0AEAMICgAwBgAEEHAMAAgg4AgAEEHQAAAwg6AAAGEHQAAAwg6AAAGEDQAQAwgKADAGAAQQcAwACCDgCAAQQdAAADCDoAAAYQdAAADCDoAAAYQNABADCAoAMAYABBBwDAAIIOAIABBB0AAAMIOgAABhB0AAAMIOgAABhA0AEAMICgAwBgAEEHAMAAgg4AgAEEHQAAAwg6AAAGEHQAAAwg6AAAGBAo9QCAJHmep2Qmqa1vbVU4EC71OEBBOmIdaqhqKPUYgCSCjjKxavEq3f839ysxkSj1KEDBVjes1vrr1suTV+pRAHnOuULXFrwQAAAUTUE/MfIcOgAABhB0AAAMIOgAABhA0AEAMICgAwBgAEEHAMAAgg4AgAEEHQAAAwg6AAAGEHQAAAwg6AAAGEDQAQAwgKADAGAAQQcAwACCDgCAAQQdAAADCDoAAAYQdAAADCDoAAAYQNABADCAoAMAYABBBwDAAIIOAIABBB0AAAMIOgAABhB0AAAMIOgAABhA0AEAMICgAwBgAEEHAMAAgg4AgAEEHQAAAwg6AAAGEHQAAAwg6AAAGEDQAQAwgKADAGAAQQcAwACCDgCAAQQdAAADCDoAAAYQdAAADCDoAAAYQNABADCAoAMAYABBBwDAAIIOAIABBB0AAAMIOgAABhB0AAAMIOgAABhA0AEAMICgAwBgAEEHAMAAgg4AgAEEHQAAAwg6AAAGEHQAAAwg6AAAGEDQAQAwgKADAGAAQQcAwACCDgCAAQQdAAADAlNY683aFAAAYEZ4hA4AgAEEHQAAAwg6AAAGEHQAAAwg6AAAGEDQAQAwgKADAGAAQQcAwACCDgCAAf8fe9hjx4aMf88AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from mmengine.visualization import Visualizer\n",
    "import numpy as np\n",
    "\n",
    "vis = Visualizer(image = np.ones((1000,1000,3))*255)\n",
    "for b in bbx:\n",
    "    vis.draw_bboxes(b+500)\n",
    "vis.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b5ec0f6-b715-4783-b9d2-4b98fe02d480",
   "metadata": {},
   "source": [
    "Anchors of different size will be assigned to feature maps of different levels."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6cb677e8-00d0-43d8-baee-31d977802721",
   "metadata": {},
   "source": [
    "We then construct an anchor-based head based on this anchor generator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "2f48c578-a2f0-49bc-a8f1-f67c5236e2b0",
   "metadata": {},
   "outputs": [],
   "source": [
    "from mmdet.models.dense_heads import RetinaHead\n",
    "abhead = RetinaHead(in_channels=256, \n",
    "                    num_classes=20,\n",
    "                    anchor_generator=dict(\n",
    "                        type='AnchorGenerator',\n",
    "                        octave_base_scale=4,\n",
    "                        scales_per_octave=3,\n",
    "                        ratios=[0.5, 1.0, 2.0],\n",
    "                        strides=[8, 16, 32, 64, 128]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "719cf78b-87e8-4ee4-86fb-1a92d764038a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from mmdet.models.dense_heads import RPNHead\n",
    "abhead = RPNHead(in_channels=256, \n",
    "                 anchor_generator=dict(\n",
    "                    type='AnchorGenerator',\n",
    "                    scales=[8, 16, 32],\n",
    "                    ratios=[0.5, 1.0, 2.0],\n",
    "                    strides=[8, 16, 32, 64, 128]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "8690c963-1d3a-4955-af3e-0f5092f67cd8",
   "metadata": {},
   "outputs": [],
   "source": [
    "out = abhead(ncout)\n",
    "cls, bbx = out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "d0fcccfa-8f88-4363-833a-1a0442ceac97",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([1, 9, 125, 75])\n",
      "torch.Size([1, 9, 63, 38])\n",
      "torch.Size([1, 9, 32, 19])\n",
      "torch.Size([1, 9, 16, 10])\n",
      "torch.Size([1, 9, 8, 5])\n"
     ]
    }
   ],
   "source": [
    "for o in cls:\n",
    "    print(o.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "cb5dac5b-5014-4c34-99a4-35d646b4fded",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([1, 36, 125, 75])\n",
      "torch.Size([1, 36, 63, 38])\n",
      "torch.Size([1, 36, 32, 19])\n",
      "torch.Size([1, 36, 16, 10])\n",
      "torch.Size([1, 36, 8, 5])\n"
     ]
    }
   ],
   "source": [
    "for o in bbx:\n",
    "    print(o.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "837d9a27-6b68-4f03-afe5-902710d4fc8d",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "e678ac14dff038bc46ab45f1b54ebd32d0ce2b34611f4e1e107e2941d3b654ca"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
